{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 Keras 做 IMDB 影评分类 - FC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import imdb\n",
    "\n",
    "# num_words = 10000 意思是只保留出现次数最多的前 10000 个词\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中每条影评存储的是各个单词在一个词表中的下标。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 14, 22, 16, 43, 530, 973, 1622, 1385, 65, 458, 4468, 66, 3941, 4, 173, 36, 256, 5, 25, 100, 43, 838, 112, 50, 670, 2, 9, 35, 480, 284, 5, 150, 4, 172, 112, 167, 2, 336, 385, 39, 4, 172, 4536, 1111, 17, 546, 38, 13, 447, 4, 192, 50, 16, 6, 147, 2025, 19, 14, 22, 4, 1920, 4613, 469, 4, 22, 71, 87, 12, 16, 43, 530, 38, 76, 15, 13, 1247, 4, 22, 17, 515, 17, 12, 16, 626, 18, 2, 5, 62, 386, 12, 8, 316, 8, 106, 5, 4, 2223, 5244, 16, 480, 66, 3785, 33, 4, 130, 12, 16, 38, 619, 5, 25, 124, 51, 36, 135, 48, 25, 1415, 33, 6, 22, 12, 215, 28, 77, 52, 5, 14, 407, 16, 82, 2, 8, 4, 107, 117, 5952, 15, 256, 4, 2, 7, 3766, 5, 723, 36, 71, 43, 530, 476, 26, 400, 317, 46, 7, 4, 2, 1029, 13, 104, 88, 4, 381, 15, 297, 98, 32, 2071, 56, 26, 141, 6, 194, 7486, 18, 4, 226, 22, 21, 134, 476, 26, 480, 5, 144, 30, 5535, 18, 51, 36, 28, 224, 92, 25, 104, 4, 226, 65, 16, 38, 1334, 88, 12, 16, 283, 5, 16, 4472, 113, 103, 32, 15, 16, 5345, 19, 178, 32]\n"
     ]
    }
   ],
   "source": [
    "print(train_data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以下载到该词表"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4822"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_index = imdb.get_word_index()\n",
    "word_index['hello']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hello'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reverse_word_index = {value: key for (key, value) in word_index.items()}\n",
    "reverse_word_index[4822]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将影评恢复为文本后，可以看出每条影评的原文。需要注意的是每个词的索引向后移动了 3 位，因为 0, 1, 2 留给了 \"padding\", \"start of sequence\", \"unknown\"。但这三个特殊字符并没有出现在单词表中，所以在解码的时候需要将索引减去 3。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_text = ' '.join([reverse_word_index.get(index - 3, '?') for index in train_data[0]])\n",
    "review_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预处理数据\n",
    "\n",
    "把评论转为 one-hot 编码的向量，以便交给神经网络来处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1\n",
    "    return results\n",
    "\n",
    "X_train = vectorize_sequences(train_data)\n",
    "X_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(32, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='nadam',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/6\n",
      "20000/20000 [==============================] - 5s 254us/step - loss: 0.3298 - acc: 0.8605 - val_loss: 0.2728 - val_acc: 0.8914\n",
      "Epoch 2/6\n",
      "20000/20000 [==============================] - 4s 192us/step - loss: 0.1652 - acc: 0.9372 - val_loss: 0.2979 - val_acc: 0.8854\n",
      "Epoch 3/6\n",
      "20000/20000 [==============================] - 4s 209us/step - loss: 0.0894 - acc: 0.9674 - val_loss: 0.3834 - val_acc: 0.8768\n",
      "Epoch 4/6\n",
      "20000/20000 [==============================] - 4s 209us/step - loss: 0.0357 - acc: 0.9882 - val_loss: 0.5003 - val_acc: 0.8654\n",
      "Epoch 5/6\n",
      "20000/20000 [==============================] - 4s 212us/step - loss: 0.0233 - acc: 0.9926 - val_loss: 0.7139 - val_acc: 0.8694\n",
      "Epoch 6/6\n",
      "20000/20000 [==============================] - 4s 210us/step - loss: 0.0150 - acc: 0.9946 - val_loss: 0.8024 - val_acc: 0.8636\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(X_train, y_train,\n",
    "                    epochs=6, batch_size=32,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 观察 history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4HNW9//H32aJuyaqWLRe5SXLDTbKptjHNEMAGHGzqDeFCCIQaCARC4JKQfhNSuCQklwskENtA4GcwhkDcQrOKLRe5yBUkF2nVrbL9/P7Y1VrVWssrrXb1fT2Pnp2dOTv7XWE+c3Rm9ozSWiOEECK8GIJdgBBCiMCTcBdCiDAk4S6EEGFIwl0IIcKQhLsQQoQhCXchhAhDEu5CCBGGJNyFECIMSbgLIUQYMgXrjVNSUnRmZmaw3l4IIUJSUVFRldY6tad2QQv3zMxMCgsLg/X2QggRkpRSX/rTToZlhBAiDEm4CyFEGJJwF0KIMNRjuCulXlJKVSqldnazXSmlfqeU2q+U2q6UmhX4MoUQQpwOf3ruLwOLTrH9cmCi9+dO4IUzL0sIIcSZ6DHctdabgJpTNFkMvKo9vgCGKqWGB6pAIYQQpy8QY+4ZQFmb5+XedUIIIYIkENe5qy7WdXnvPqXUnXiGbhg9enQA3loIMZBprcHtBpcL7f3B6US73WinE1wucLs97Vrbao12a9Cty57H1u2ebd7t/rzWrYHW17o96aTdJ/fbbvvJffvaut2e561tO+7b196P17rdgCbuwguJnjatT3/3gQj3cmBUm+cjgaNdNdRavwi8CJCbmys3bxWDnna5cDU04G5owFVfj9tq9QahG1zOk4HocqGdLu86N9rl9AWmb5vb8+jZ5vZuc3rWuV3g9Lbvrl3rfp1t9uvq0M4bzDg71NY2tDu0Ex0ohSltWEiE+2rgO0qpFcBcoF5rfSwA+xUiZLitVlz19Z6A9j56fhq8j3WeAK/zrm8N84aGvivKZEIZDJ5Ho7HdMiYjymD0Lrdvh9GAMnpfExmBwRDtaW80oYwGMHZoZzKC0bO/HtsZjCfbG02ebQYDKAMYFEopaH2uOLlNKc92g3dZtV0+xWsNBkChDK3bPO2VQXn32dX2Nvs2GDz77W7fqu37dt53u9e2Pu8nPYa7UurvwAIgRSlVDjwFmAG01n8E3geuAPYDzcBtfVWsEH1Ju924T5zoEMx1vhD2BXN9Pa4Gb4jXeYJa22zd79hoxBgfjzEhwfOTnETEuHGe5fh4jEM96w3x8Riio72B2D4MPeuMJ7cZDZ7nHQPc266/g0QMPD2Gu9b6hh62a+CegFUkxBly22yn3YP29aJ196OFKibmZCAnJBCRORbjUE8oGxOGesP7ZIgb4hM822NjJWhFvwvaxGFCnIrWuk0v2tODdreGcccedIeg1lZr9zs2GHwBbRiagDFxKBFjxpzsVSfEY/Att/mJj0dFRPTfL0CIMyThLvqV1hpXXR3OykqclRbPo6X10bPOYanEZalCOxzd7kdFR7cL3ojMMZ5Qjk9oF9TGhIR2YW2IjfUMYwgR5iTcRUCcDG1vULcLbc/jqULbkJCAOS0VU2oasZlzMKWlYUxO6nq4IyEBg/SihTglCXdxSp1C29JNeFssPYR2qi+0Tampnse0VN9zQ2RkED6dEOFLwn2Q6ja0O4b3KULblJqCOS2NmMw8T1CntgnstDRMKSkYoqKC8OmEEBLuYaZdaHfXy/Y+dhna8fGY0lK9oZ3boaftXU5NldAWYoCTcA8RXYZ2N+HdbWinpmJKSyUmL7dzYLcOj0hoCxEWJNwHKFdjEy1bimjavJnmzfnYSkvRdnundoYhQ3zj19G5szF3DGwJbSEGJQn3AcLd0kLL1q00bc6nefNmWnbs8EyqZDYTfdZZJN50E+b0YZ172tHRwS5dCDEASbgHidtmo6V4G82bN9OUvxnrtu2e4RSjkeipU0m+/XZi5s4hZuZMDDExwS5XCBFiJNz7ibbbadm5k6YvvqB5cz4txcWe+UgMBqImTybx1luInTuX6FmzMcbFBrtcIUSIk3DvI9rpxFpS4htmad6yBd3SAkBkTg6Jy5cRM/dsYnJnY4yPD3K1QohwI+EeINrlwrp7j2+YpaWwCHdTEwCREycw9NprPcMseXmYEhODXK0QItxJuPeSdrux7dvnCfPN+TQXFPjm5o7IzCT+qiuJnTuXmDlzMCUnB7laIcRgI+HuJ6019gMHfJcmNufn46qrA8A8ahRDLr3EG+ZzMQ9LC3K1QojBTsK9G1prHF9+6R0z/4Km/AJcVVUAmEYMJ27BAmLmziV27hzMI0YEuVohhGgv9MJ997tQ/DoYTGA0g8EMRpP30QzGiO63tVtvbr9sMGG31NO8Yz9N20tpLt6Ns6oGAFNKErG5M4nNnUXMnDzMo0ajTBGe1xqMQf6FCCFEZ6EX7rYTUFcGbge4HN5H58nnvnUOPLcf756jyUBTZSTNlZE0V0TgaPb8OoyRLmLS7MTm2ohJsxEx5ChK7YT9f/XcTLAjg7mbg0Z3BxaT9yDUsU03r2k9YJmiwBTpeW6KAlMEGCM960yR3mXvNmOEd32bZYPJcz9HIUTYC71wn3Gj58cfble7sHdWVtCUX0hz0RaaCotxHDkOgHFIHDGzskmaNpHYaeOJyEhFaWfng0XHA4nb2X57u7bO7l/jtIGtsYf9ta63g3YH6JenOhwUujpARHQ4WHQ4cHRa5+frTFHt92E0y4FGiD4UeuF+Gpx19TTn5/tOgtoPHgQ887HE5OWRdOs3iJk7l8isrIF9dx632xPyLhs4Wx+9P63rnFZPG6etw7Ktw+usXezD+/rW11rru9+36xQ3gj4tqucDhDkKTNFgjgZzjOd567Ipyrsuuv1Pp/YxJ9cbw/qfuxDthNW/dlddHU0FBZ6rWTZvxrZvHwCGmBiic2cz9LpriZkzl6jJkzx3iQ8VBgMYojxhFWxatwn6NgcF37K9w0GhmwNEpwNLFwctezM0V4OjBRxWcDR7lp0tvavdYG5zQOgQ/B0PEp0OIF2173gA8S6bIsP7rxKtPX9Nul2evza1y7vs8i472yy7TtHOu95g9AwZdjwn1u25M7Pn/wlxSiEd7q4TJ2guLKR5cz5N+Zux7d4DWqOiooiZNYv4K68kdu4coqZMQZnNwS43PCh1sncdLFp7DgiOljY/zd51zd2sbzn1tuaqDgcQK9ibPCF02lQXB4lo/PpLQxm6DsAzDVDt7vAap+cvwnavcXYR2t52Hd8z6FT3wd/lua7TOXh0bHuKizLa7d/ftmaIHNLnnbWQC/eWkhJOrF1L0+Z8rCUl4HajIiKInjGDlO/cQ+zZZxM9bZrcqT6cqTbh2ddcjlMcQDr8NeE7gHQ4SLQ9qNiboKmqc/tT/TXSGjbK6O3lGtssd7Xe5P1rz9S+ncHU5sR6m3a+ZWP7fXW7b2P36/2qrcPrtbvDuacezm25nW3WdbGtu324nZ7fuT/nulrX9XBRRq997b8h7z/7Zt9eoRfuRUVUv/Iq0WedRcpd3yJmzlyiZ86Qe3CKvtF69VNUH8//43Z7DgTa3SEAZfghqDpclNHjwaPLbV20HX1un5cecuGecO11DF26VKbBFeHFYIAI+Tc94LT+dcEAON91mkIu3GU6XCGE6Jn8zSeEEGFIwl0IIcKQhLsQQoQhCXchhAhDEu5CCBGGJNyFECIM+RXuSqlFSqm9Sqn9SqnHutg+Wim1Xim1VSm1XSl1ReBLFUII4a8ew10pZQSeBy4HJgM3KKUmd2j2A2CV1nomsBz4n0AXKoQQwn/+9NznAPu11ge11nZgBbC4QxsNtH4/OwE4GrgShRAi9H1V3czLnx7i1pfy+fc+S5+/nz/fUM0Ayto8LwfmdmjzNPBPpdS9QCxwcUCqE0KIEOVwuSk4XMP6PZWs21PJAUsTAONSYmmyOfv8/f0J964mpu44VdoNwMta6/9WSp0D/FUpNVXr9rcQUkrdCdwJMHr06N7UK4QQA1ZVo40Ney2s21PBv0urOGFzYjYqzh6XzE1zx7AwJ43MlP6ZQsWfcC8HRrV5PpLOwy63A4sAtNafK6WigBSgsm0jrfWLwIsAubm5fTSXphBC9A+3W1NytIF1eypZt7eS7eV1aA1pQyK5YtpwLsxJ4/yJKcRF9v80Xv68YwEwUSk1FjiC54Rpx5uYfgVcBLyslJqEZwq1vh9UEkKIftZoc/LJPgvr9lSyfq8FywkbSsH0kUN58OIsFuakMXl4PAZDcO/G1WO4a62dSqnvAB8CRuAlrXWJUuoZoFBrvRr4LvBnpdSDeIZsvqG1lp65ECIsHKpq8vTO91SQf6gGh0szJNLEvOxUFmanMT87lZS4gXVPCRWsDM7NzdWFhYVBeW8hhDgVu9NN/qEab++8kkNVnpOhE9LiWJiTxoXZaeRmJmI29v/3QJVSRVrr3J7ahdx87kII0RcqG6ys3+u5suWTfVU02V1EmAycMy6Zb5ybycKcNEYlhc4NVSTchRCDktut2VZe57lUcW8lO480ADA8IYrFMzNYmJ3GuROSiYkIzZgMzaqFEKIXGqwO/l1axbo9lWwsraSq0Y5BwazRiTxyWTYLc9LISR+CUsE9GRoIEu5CiLClteaApdF7MrSSwsO1ON2ahGgz87NSWZiTxvysVBJjI4JdasBJuAshworV4eKLg9W+4ZaymhYActKHcMe8cSzMSWPmqKGYgnAytD9JuAshQt6x+hbW7/Fce/7p/ipaHC6izAbOG5/Ct+aN58KcNDKGRge7zH4l4S6ECDkut6a4rNY73GJh9zHPydCModF8PXckF+akcc64ZKLMxiBXGjwS7kKIkFDXbGdjqYX1eyrZWGqhttmB0aCYPSaRxy7PYWFOGhPT4sLiZGggSLgLIQYkrTWlFY2+b4YWfVmLW0NSbAQXZqdxYU4a8yamkhBjDnapA5KEuxBiwLA6XHx2wHOp4vo9Fo7UeU6GThkRzz0XTuDCnDSmjxyKMcjztoQCCXchRFCV1zb75jz/7EA1NqebmAgj501I4TsLJ3BhdhrpCVHBLjPkSLgLIYKitOIEj761na1f1QEwJjmGG+aMZmFOGnPHJRFpGrwnQwNBwl0I0a+01rz82WF+unYP8VEmnrhiEgsnpTEuJVZOhgaQhLsQot9UnrDyyBvb2VhqYWFOGr9YetaAmyo3XEi4CyH6xUe7Knj0re002538aMlUbp47WnrqfUjCXQjRp5rtTn68Zjevb/6KKSPi+e3yGUxIGxLsssKehLsQos/sKK/n/hVbOVTdxLfmj+O7l2QTYQrvOV0GCgl3IUTAudyaP248wG8+KiV1SCSv/edczh2fEuyyBhUJdyFEQJXXNvPQqm3kH6rha2cN5ydLpsm3SINAwl0IETD/r/gIP3hnJ1rDf399OtfOypCTpkEi4S6EOGMNVgc/fGcn7xQfZfaYRJ5bNiOk7jcajiTchRBnJP9QDQ+uLOZ4g5WHLsni7gXjw/5GGKFAwl0I0SsOl5vnPi7lhQ0HGJUUwxt3ncOs0Yk9v87hoLy8HKvV2g9Vhq6oqChGjhyJ2dy78xUS7kKI03aoqokHVmxlW3k91+eO5IdXTSEu0r84KS8vZ8iQIWRmZsp4fDe01lRXV1NeXs7YsWN7tQ8JdyGE37TWrCgo45l3dxFhMvDCTbO4fNrw09qH1WqVYO+BUork5GQsFkuv9yHhLoTwS02Tncfe2s4/d1Vw3oRk/vvrM3o9Fa8Ee8/O9Hck4S6E6NGmUgsPv7GNumYHT1wxidvPH4tBbpgxoMkpbSFEt6wOF//1bgm3vpRPQrSZd+45jzvmjRtUwR4XF9fttsOHDzN16tR+rMZ/0nMXQnRpz/EG7v97MXsrTvCNczN57PIcosxyA41QIeEuhGjH7db832eH+fkHe4iPMvN/t+VxYXZan7zXf71bwq6jDQHd5+QR8Tx11ZRutz/66KOMGTOGu+++G4Cnn34apRSbNm2itrYWh8PBj3/8YxYvXnxa72u1Wvn2t79NYWEhJpOJX//611x44YWUlJRw2223YbfbcbvdvPXWW4wYMYLrr7+e8vJyXC4XTz75JMuWLTujz92RhLsQwqeiwcrDb2zj3/uquHhSGj+7LvxuprF8+XIeeOABX7ivWrWKDz74gAcffJD4+Hiqqqo4++yzufrqq0/rpObzzz8PwI4dO9izZw+XXnoppaWl/PGPf+T+++/npptuwm6343K5eP/99xkxYgRr1qwBoL6+PuCf069wV0otAn4LGIG/aK1/1kWb64GnAQ1s01rfGMA6hRB97MOS4zz21nZaHC6evWYqN87p+5tpnKqH3VdmzpxJZWUlR48exWKxkJiYyPDhw3nwwQfZtGkTBoOBI0eOUFFRQXp6ut/7/eSTT7j33nsByMnJYcyYMZSWlnLOOefw7LPPUl5ezrXXXsvEiROZNm0aDz/8MI8++ihXXnklF1xwQcA/Z48nVJVSRuB54HJgMnCDUmpyhzYTge8D52mtpwAPBLxSIUSfaLI5eeyt7Xzrr0VkJEbz3r0XcNPcMWF9ueLSpUt58803WblyJcuXL+e1117DYrFQVFREcXExw4YNO+1v0Gqtu1x/4403snr1aqKjo7nssstYt24dWVlZFBUVMW3aNL7//e/zzDPPBOJjteNPz30OsF9rfRBAKbUCWAzsatPmDuB5rXUtgNa6MtCFCiECb1tZHQ+sLOZwdRPfXjCeBy/OGhQ301i+fDl33HEHVVVVbNy4kVWrVpGWlobZbGb9+vV8+eWXp73PefPm8dprr7Fw4UJKS0v56quvyM7O5uDBg4wbN4777ruPgwcPsn37dnJyckhKSuLmm28mLi6Ol19+OeCf0Z9wzwDK2jwvB+Z2aJMFoJT6FM/QzdNa6w8CUqEQIuBcbs0LG/bz3Mf7SBsSyd/vOJuzxyUHu6x+M2XKFE6cOEFGRgbDhw/npptu4qqrriI3N5cZM2aQk5Nz2vu8++67ueuuu5g2bRomk4mXX36ZyMhIVq5cyd/+9jfMZjPp6en88Ic/pKCggEceeQSDwYDZbOaFF14I+GdU3f0p4Wug1NeBy7TW/+l9fgswR2t9b5s27wEO4HpgJPBvYKrWuq7Dvu4E7gQYPXr07N4cHYUQZ6asppmHVhVTcLiWq6aP4MeLp/brzTR2797NpEmT+u39QllXvyulVJHWOren1/rTcy8HRrV5PhI42kWbL7TWDuCQUmovMBEoaNtIa/0i8CJAbm7uqY8qQoiAe2frEZ58Zyca+M2y6SyZITfTCFf+hHsBMFEpNRY4AiwHOl4J8w5wA/CyUioFzzDNwUAWKoTovfoWB0++s5PV246SOyaR38jNNE7Ljh07uOWWW9qti4yMZPPmzUGqqGc9hrvW2qmU+g7wIZ7x9Je01iVKqWeAQq31au+2S5VSuwAX8IjWurovCxdC+GfzwWoeWrWN4w1WvntJFt+Wm2mctmnTplFcXBzsMk6LX9e5a63fB97vsO6HbZY18JD3RwgxANidbn7zcSl/3HiAMUkxvHnXOcz042YaIjzIN1SFCEMHLI08sKKYHUfqWZY7ih9eNZlYP2+mIcKD/NcWIoxorXk9/yt+/N5uIs0G/njzbBZN9f9bliJ8SLgLESaqG208+tYOPt5dwQUTU/jV16czLL53N9MQoU/CXYgwsGFvJY+8uZ36ZgdPXjmZ287NHFRzrveluLg4Ghsbg13GaZNwFyKEWR0ufrZ2Dy9/dpjsYUN49ZtzmDQ8PthliQFAwl2IELXraAMPrNxKaUUjt52XyaOLQvBmGmsfg+M7ArvP9GlweaeJa30COZ97Y2Mjixcv7vJ1r776Kr/61a9QSnHWWWfx17/+lYqKCu666y4OHvR8DeiFF17g3HPPDcCH7kzCXYgQ43ZrXvr0EL/4YC8JMWZe+eYc5melBruskBHI+dyjoqJ4++23O71u165dPPvss3z66aekpKRQU1MDwH333cf8+fN5++23cblcfTrcI+EuRAg5Xu+5mcYn+6u4ZPIwfnbtNJJD+WYap+hh95VAzueutebxxx/v9Lp169axdOlSUlJSAEhKSgJg3bp1vPrqqwAYjUYSEhL67HNKuAsRIj7YeYzH/rEDm8PNT66Zxg1zRsm8ML3UOp/78ePHO83nbjabyczM9Gs+9+5ep7UO+n8b+Q6yEANck83J997cxl1/28KoxBjW3Hc+N87t+7skhbPly5ezYsUK3nzzTZYuXUp9fX2v5nPv7nUXXXQRq1atorraMwtL67DMRRdd5Jve1+Vy0dAQ2PvHtiXhLsQAtvWrWq743b95o6icuxeM561vn8u41LhglxXyuprPvbCwkNzcXF577TW/53Pv7nVTpkzhiSeeYP78+UyfPp2HHvLMzPLb3/6W9evXM23aNGbPnk1JSUmffcYe53PvK7m5ubqwsDAo7y3EQOd0ufmfDQf47b/2kR4fxa+vn87cMLmZhszn7r++ns9dCNGPymqaeWBlMUVf1rJ4xgieWTyVhOj+u5mGCA8S7kIMEFpr/rHlCE+tLkEBzy2bwZKZGcEuSxCm87kLIfqWy63ZtM/Cq58dZv1eC3mZifz6ermZxkAStvO5CyECr6ymmVWFZbxZVM6xeitJsRF8b1E235o3HqPMCyPOkIS7EP3I6nDxYclxVhaU8dmBapSCeRNTefLKyVw8aRgRJrmATQSGhLsQ/aDkaD2rCsp4p/go9S0ORiZG89AlWSydPZIRQ6ODXZ4IQxLuQvSR+hYHq7cdZWXBV+w80kCE0cBlU9NZljuKc8cny5S8QRSq0/ieDgl3IQJIa80XB2tYVVjG+zuOYXO6mTQ8nqevmsySmRkMjYkIdolikJBwFyIAKhqsvFlUzqrCMr6sbmZIpImls0eyPG80UzPiZaqAAUprzfe+9z3Wrl2LUoof/OAHLFu2jGPHjrFs2TIaGhpwOp2+qXlvv/12CgsLUUrxzW9+kwcffDDYH6FbEu5C9JLD5WbdnkpWFZSxfm8lbg1zxyZx/0UTuXzqcKIjQmxu9SD4ef7P2VOzJ6D7zEnK4dE5j/rV9h//+AfFxcVs27aNqqoq8vLymDdvHq+//jqXXXYZTzzxBC6Xi+bmZoqLizly5Ag7d+4EoK6uLqB1B5qEuxCn6YClkVWFZbxVdISqRhtpQyL51vzxXJ87irEpscEuT5yGTz75hBtuuAGj0ciwYcOYP38+BQUF5OXl8c1vfhOHw8GSJUuYMWMG48aN4+DBg9x777187Wtf49JLLw12+ack4S6EH5rtTtZsP8aqwjIKDtdiNCgW5qSxLHcUC7JTMRnlEsbe8LeH3Ve6m1tr3rx5bNq0iTVr1nDLLbfwyCOPcOutt7Jt2zY+/PBDnn/+eVatWsVLL73UzxX7T8JdiG5ordlWXs/KgjLe3XaURpuTsSmxPLooh+tmZ5A2JCrYJYozNG/ePP70pz/xH//xH9TU1LBp0yZ++ctf8uWXX5KRkcEdd9xBU1MTW7Zs4YorriAiIoLrrruO8ePH841vfCPY5Z+ShLsQHdQ02Xl76xFWFZSxt+IEUWYDX5s2gmV5o8jLTJSTo2Hkmmuu4fPPP2f69OkopfjFL35Beno6r7zyCr/85S8xm83ExcXx6quvcuTIEW677TbcbjcAP/3pT4Nc/anJlL9C4Lkv6Sf7q1hZWMZHJRXYXW6mj0zg+rxRXDV9BPFRMitjoMiUv/6TKX+F6KXy2mbeKCznzaJyjtS1MDTGzE1nj2ZZ3ihy0uODXZ4QvSbhLgYdm9PFR7sqWFlQxif7qwA4f0IK378ih0smDyPSJJcwitAn4S4GjT3HG1hZUMY7W49Q2+wgY2g09y2cyNdzRzIyUabXFeFFwl2EtRNWB+9uO8bKwjK2ldVhNiounZzOsrxRnDchRabWFWFLwl2EHa01BYdrWVngmd+lxeEie9gQnrxyMtfMzCApVuZ3EeHPr3BXSi0CfgsYgb9orX/WTbulwBtAntZaLoUR/aryhJV/bPFcwniwqom4SBNLZo7g+txRzBg1VC5hFINKj+GulDICzwOXAOVAgVJqtdZ6V4d2Q4D7gIF7U0ERdpwuNxtLLawoKGPdnkpcbk1eZiLfXjCer501nJgI+eNUDE7+/MufA+zXWh8EUEqtABYDuzq0+xHwC+DhgFYoRBcOVzX5blFXecJGSlwE/3nBWK7PHcX41LhglydE0PkT7hlAWZvn5cDctg2UUjOBUVrr95RS3Ya7UupO4E6A0aNHn361YlCzOlys3XmMFfllbD5Ug0HBhdlpXJ83ioU5aZhlfhfhpyVLllBWVobVauX+++/nzjvv5IMPPuDxxx/H5XKRkpLCv/71LxobG7n33nt90/w+9dRTXHfddcEu3y/+hHtXA5W+r7UqpQzAb4Bv9LQjrfWLwIvg+YaqfyWKwUxrzc4jDaws/Ir/V3yUE1YnY5JjeOSybK6bNZL0BJnfJZQd/8lPsO0O7JS/kZNySH/88VO2eemll0hKSqKlpYW8vDwWL17MHXfcwaZNmxg7diw1NTUA/OhHPyIhIYEdO3YAUFtbG9Ba+5I/4V4OjGrzfCRwtM3zIcBUYIP3hFU6sFopdbWcVBW9VdNk591tR1lRUMbuYw1EmgxcMW041+eOYu7YJLlFnTgjv/vd73j77bcBKCsr48UXX2TevHmMHTsWgKSkJAA+/vhjVqxY4XtdYmJi/xfbS/6EewEwUSk1FjgCLAdubN2ota4HUlqfK6U2AA9LsIvT4XJrtpfXsWGvhY2lFraV16E1TMtI4EdLpnL19BEkRMv8LuGmpx52X9iwYQMff/wxn3/+OTExMSxYsIDp06ezd+/eTm211iF7lVWP4a61diqlvgN8iOdSyJe01iVKqWeAQq316r4uUoQnywkb/95nYcNeC//eZ6G22YFSMGPUUO6/aCKXTk5n8giZ30UEVn19PYmJicTExLBnzx6++OILbDYbGzdu5NChQ75hmaSkJC699FL+8Ic/8NxzzwGeYZlQ6b3LrJCi3zhdborL6thY6gn0HUfqAUiJi2BeVioLstO4YEIKifIlo7AW7FkhbTYbS5Y15SJqAAASTElEQVQs4ciRI2RnZ2OxWHj66adpaWnh8ccfx+12k5aWxkcffURjYyP33HMPRUVFGI1GnnrqKa699tp+q1VmhRQDVmWDlQ2lFjZ6e+cNVidGg2LW6KE8fGkWC7LTmDw8XsbQRb+JjIxk7dq1XW67/PLL2z2Pi4vjlVde6Y+yAk7CXQSUw+Wm6MtaX+9897EGANKGRLJoajrzs9I4f0IKCTEyfi5EX5JwF2fsWH2L50ToXguf7q/ihM2JyaCYPSaRRxflMD8rlUnDh4TsiSkhQpGEuzhtNqeLosO1vuGWvRUnABieEMWV04czPyuN8yYkM0TuXiS6EcpXofSXMz0fKuEu/FJW0+wbavnsQBXNdhdmo2LO2CSWzp7E/OxUJqbFyf+wokdRUVFUV1eTnJws/166obWmurqaqKjef0lPwl10yepwkX+oxnvdeSUHLE0AjEyM5tpZGSzISuOc8cnERso/IXF6Ro4cSXl5ORaLJdilDGhRUVGMHDmy16+X/zOFz5fVTWzYa2HD3ko+P1iN1eEmwmTg7HHJ3Dh3DAuyUxmXEiu9LXFGzGaz75ugou+EXLgfbzrOlootLBq7CIOSiaLORIvdxRcHq73DLZUcrm4GIDM5huV5o5mfncrZY5OJjpB7igoRakIu3N8sfZM/bf8Tr+x6he/O/i5zhs8JdkkhQ2vNwaom31f8vzhYjd3pJsps4Jxxydx23ljmZ6WSmRIb7FKFEGco5ML97hl3MyZ+DL/f+ntu/+ftnJ9xPg/OfpCsxKxglzYgNdmcfH6gmg2llWwstVBW0wLA+NRYbjl7DPOzUpkzNokos/TOhQgnITv9gM1l4++7/86LO16k0d7I4gmLuWfGPaTHpgewytCjtWZfZSMb91rYUFpJwaFa7C43MRFGzh2fwoLsVOZnpTIqKSbYpQohesHf6QdCNtxb1dvq+cuOv/Da7tcwKAM3T7qZ26fdzpCIIQGoMjScsDr4dH81G0sr2bjXwtF6KwDZw4YwPzuVBVmpzM5MJNIkvXMhQt2gCfdWRxqP8Ietf2DNwTUkRCZw51l3six7GRHG8JuESmvNnuMnfFe2FH1Zi9OtiYs0cf6EFOZ7e+cjhkYHu1QhRIANunBvtbt6N78p+g2fH/ucjLgM7pt5X1hcWVPf4uCTfVWe3nmphYoGGwCThsezwNs7nzUmUW41J0SYG7Th3uqzI5/x66Jfs7d2L5OTJw/IK2ta7C6qm2xUN9qpabJT3WSnxvvcs+x5rG60cazeisutiY8yccHEVF/vfFi83GZOiMFk0Ic7gFu7WXNwDb/b+juONx3ngowLeGD2A312ZU2z3dkmmNuHtmfZRk2TnSrv+haHq8v9RJgMJMdGkOT9SYmLZFRiNPOyUpkxaigm6Z0LMWhJuLfRmytrtNY0213eMLa16Vl7etKtyzXe4K5usmF1uLvcV4TJQEpsBElxESTFRnqWvc+TYyNIjo30LSfFRhAXaZJvgQohuiTh3oHWmqMnanhx259ZfWgVCkVe0mImxSyhsdnUZXjbnF2HdaTJQEpcpK9nnRwbQbI3uE8unwzt2AijhLUQIiDC/k5MWmsabc5Owx7VTXZqvEMjrcMjNY12qprs2J1uYBrKnEFk6kd8pt/gk4o16NqLGeqcT0pcDMlxEUwcFtdFeEf6etYxEtZCiAEu5ML99c1f8ft1+6hutGN3dd2zjjYbvWPVEaTGRZI9LJ7kNsMenuWrqHUe4q+l/0O+aTXxcUV8e9b9XJZ5WchfWSOEECE3LLN+byVrth/rNBRyMrQjT3uiq1C4skYIIUDG3E9bf19ZI4QQveFvuMv4g5dBGbhq/FW8d817fHf2dym2FLN09VKe/PRJjjcdD3Z5QghxWqTn3o16Wz1/3v5nXt/z+qCds0YIMfBIz/0MJUQm8HDew7x7zbtcMuYS/nfn/3LFP67gb7v+ht1lD3Z5QghxShLuPciIy+CnF/yUVVeuIicph58X/Jyr37matYfW4tZdX60jhBDBJuHup0nJk/jzpX/mTxf/iThzHN/b9D1uWHMD+cfyg12aEEJ0IuF+ms7NOJdVV63iJ+f/hBprDbf/83bu/vhuSmtLg12aEEL4SLj3glxZI4QY6ORqmQCQK2uEEP1FrpbpR3JljRBioPEr3JVSi5RSe5VS+5VSj3Wx/SGl1C6l1Hal1L+UUmMCX+rAJ1fWCCEGih7DXSllBJ4HLgcmAzcopSZ3aLYVyNVanwW8Cfwi0IWGkq6urLlxzY1yZY0Qot/403OfA+zXWh/UWtuBFcDitg201uu11s3ep18AIwNbZmg6N+NcVl65kmfPf5Zqa7VcWSOE6Df+hHsGUNbmebl3XXduB9aeSVHhxGgwcvX4q3nvmvd4aPZDcmWNEKJf+BPuXd2VostLbJRSNwO5wC+72X6nUqpQKVVosVj8rzIMRBojuW3qbay9di23Tr6VNQfXcOXbV/Jc0XOcsJ8IdnlCiDDjT7iXA6PaPB8JHO3YSCl1MfAEcLXW2tbVjrTWL2qtc7XWuampqb2pN+TJlTVCiP7gT7gXABOVUmOVUhHAcmB12wZKqZnAn/AEe2Xgyww/rVfWrLxypVxZI4QIuB7DXWvtBL4DfAjsBlZprUuUUs8opa72NvslEAe8oZQqVkqt7mZ3ooPJyZPlyhohRMDJN1QHEJfbxZpDa/j91t/L3aCEEF2Sb6iGoE5X1lSevLLmaGOn0xxCCNEt6bkPYG3nrHG4HYxLGEdeeh656bnkDsslJTol2CUKIfqZ3CA7jBxtPMoHhz+g4HgBWyq20Oz0fF9sfMJ4ctNzyUvPIy89j6SopCBXKoToaxLuYcrhdrC7ejcFxws8YV+5hRZnCwAThk4gd1guc4bPIXdYLolRiUGuVggRaBLug4TD7WBX9S5f2G+t3OoL+4mJE8kb5unV5w7LZWjU0CBXK4Q4UxLug5TD7aCkqsQX9sWWYl/YZyVmeYZwhnnG7RMiE4JcrRDidEm4CwAcLgc7q3eeDPvKYqwuKwp1MuzT85g9bLaEvRAhQMJddMnusrOzame7nr3NZUOhyE7K9vXsZw2bJWEvxAAk4S78YnfZ2VG1g/zj+RQeL6S4shi7245CkZOU4+vZzxo2i/iI+GCXK8SgJ+EuesXmsrHdsp3C44UUVBSwrXIbdrcdgzJ4wn5YHnOGz2Fm2ky5R6wQQSDhLgKiNewLjheQfzyf7ZbtONwODMrApKRJzEmfQ256LrPSZhEXERfscoUIexLuok9YnVa2W7aTfzyfguMFbK/ajtPtxKiMTE6e7PlSlXfMPtYcG+xyhQg7Eu6iX7Q4W3xhX3i8sF3YT0me4vsG7ay0WcSYY4JdrhAhT8JdBEWLs4XiymIKjhdQWFHIDssOnNob9ilTPGP26XOYkTZDwl6IXpBwFwNCs6OZYksxhccLyT+eT0lVCU7txKRMTEmZ4huzn5EqYS+EPyTcxYDU7GimuLLYM2ZfUUBJVQku7cJkMDEtZRq5wzzDODPSZhBtig52uUIMOBLuIiQ0OZrYWrnVM4xzvJCS6pNhPylpElmJWUxMnOh5HDpR5scRg56EuwhJTY4mtlRsoeB4ASXVJZTWllJnq/NtT4tOY2LSRLKGngz9cQnjMBvNQaxaiP4j4S7CgtaaqpYqSmtL2Ve7z/NYt48DdQdwuB0AmJSJzIRMX9i3/gyLGYZSKsifQIjA8jfcTf1RjBC9pZQiNSaV1JhUzss4z7fe4XbwVcNXlNaW+oK/uLKYtYfW+toMiRjCxKET2w/tJE6U6+/FoCA9dxFWGuwN7K/d7+vlt/b0mxxNvjYZcRmdAn/0kNGYDNLXEQOf9NzFoBQfEc+sYbOYNWyWb53WmqNNR08O63gfN5ZvxK3dAEQaIxmXMM43pNMa/MnRycH6KEKcEem5i0HL5rJxsO5gp/H8qpYqX5ukqKR2YZ+VmMX4oeOJNEYGsXIxmEnPXYgeRBojmZQ8iUnJk9qtr7HWtB/Wqd3Hqr2rsLlsABiUgTHxYzqN54+IG4FBGYLxUYToRHruQvjB5XZRdqLM17svrfEEf3ljua9NrDmWCUMndBrPl3nwRSDJpZBC9IMmRxP76/Z36uk32Bt8bdJj031fwmod2hmTMAazQa7NF6dPhmWE6Aex5limp05neup03zqtNRXNFe3G8UtrS/ns6Gc43U4AzAYz4xLGtevhZyVmkRqdKtfmi4CQnrsQ/cThcnCo4VD7E7i1+6horvC1USiiTFFEm6J9P1HGKKLN3kdTdOft3uet7aKNHda3aR9ljMJoMAbxtyDOlPTchRhgzEazb1imrXpbvS/sa6w1tDhbsDqtnkeXlWZnM1anlWpr9cn1bbafrghDRLuDRaeDRNuDxSm2tzv4eNfHmGIwGUzy18cAIOEuRJAlRCaQm55LbnqPnbFO3NqN1WnF6mof+q0/HQ8GLa6u11udVhrtjVhaLLQ4Wnz7a3G2+L4L4C+jMnZ9MOjwF0XrcqQxkghjBGaDmQhjBBGGCM9zo5kIQ0S329ut866Xg8pJEu5ChDCDMhBjjumzufC11jjcjs4HC5eVFkf3Bwtfe2+71oNFQ3NDu+0tzhbfHEGBYDKYfAeE1oND2/DvuK6rA0a7A0yHg0enfbZ9bev2NuuCeWmsX+GulFoE/BYwAn/RWv+sw/ZI4FVgNlANLNNaHw5sqUKI/qaU8oVYQmRCn7xH6wHE7rJjc9l8y3aXHbvb89jVup62t1vXZn2Ts4k6W90pXx8oJmXqFPhmg5l7ZtzDorGLAvY+Xb53Tw2UUkbgeeASoBwoUEqt1lrvatPsdqBWaz1BKbUc+DmwrC8KFkKEl7YHkDjigl1Ou4ONL/xdjpMHlTbrbC5btweIdq/xbre5bNhdduIj+/67D/703OcA+7XWBwGUUiuAxUDbcF8MPO1dfhP4g1JK6WBdiiOEEL3U9mATyvwZEMoAyto8L/eu67KN1toJ1AMy45IQQgSJP+He1ennjj1yf9qglLpTKVWolCq0WCz+1CeEEKIX/An3cmBUm+cjgaPdtVFKmYAEoKbjjrTWL2qtc7XWuampqb2rWAghRI/8CfcCYKJSaqxSKgJYDqzu0GY18B/e5aXAOhlvF0KI4OnxhKrW2qmU+g7wIZ5LIV/SWpcopZ4BCrXWq4H/Bf6qlNqPp8e+vC+LFkIIcWp+XeeutX4feL/Duh+2WbYCXw9saUIIIXpL7iwghBBhSMJdCCHCUNCm/FVKWYAve/nyFKCqx1bhRT7z4CCfeXA4k888Rmvd4+WGQQv3M6GUKvRnPuNwIp95cJDPPDj0x2eWYRkhhAhDEu5CCBGGQjXcXwx2AUEgn3lwkM88OPT5Zw7JMXchhBCnFqo9dyGEEKcQcuGulFqklNqrlNqvlHos2PX0NaXUS0qpSqXUzmDX0l+UUqOUUuuVUruVUiVKqfuDXVNfU0pFKaXylVLbvJ/5v4JdU39QShmVUluVUu8Fu5b+oJQ6rJTaoZQqVkoV9ul7hdKwjPeuUKW0uSsUcEOHu0KFFaXUPKAReFVrPTXY9fQHpdRwYLjWeotSaghQBCwJ8//OCojVWjcqpczAJ8D9Wusvglxan1JKPQTkAvFa6yuDXU9fU0odBnK11n1+XX+o9dx9d4XSWtuB1rtChS2t9Sa6mD45nGmtj2mtt3iXTwC76XyDmLCiPRq9T83en9DpefWCUmok8DXgL8GuJRyFWrj7c1coEUaUUpnATGBzcCvpe94himKgEvhIax3un/k54HuAO9iF9CMN/FMpVaSUurMv3yjUwt2vOz6J8KCUigPeAh7QWjcEu56+prV2aa1n4LkhzhylVNgOwymlrgQqtdZFwa6ln52ntZ4FXA7c4x127ROhFu7+3BVKhAHvuPNbwGta638Eu57+pLWuAzYAi4JcSl86D7jaOwa9AliolPpbcEvqe1rro97HSuBtPEPNfSLUwt2fu0KJEOc9ufi/wG6t9a+DXU9/UEqlKqWGepejgYuBPcGtqu9orb+vtR6ptc7E8//xOq31zUEuq08ppWK9FwiglIoFLgX67Cq4kAp3rbUTaL0r1G5glda6JLhV9S2l1N+Bz4FspVS5Uur2YNfUD84DbsHTmyv2/lwR7KL62HBgvVJqO55OzEda60FxeeAgMgz4RCm1DcgH1mitP+irNwupSyGFEEL4J6R67kIIIfwj4S6EEGFIwl0IIcKQhLsQQoQhCXchhAhDEu5CCBGGJNyFECIMSbgLIUQY+v901POfuUDj4AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "%matplotlib inline\n",
    "\n",
    "pd.DataFrame(history.history).plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从第 1 个 epoch 起，验证集上的 loss 就一直上升，可见模型过拟合了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用 Dropout 对抗过拟合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/6\n",
      "20000/20000 [==============================] - 5s 239us/step - loss: 0.3734 - acc: 0.8448 - val_loss: 0.2810 - val_acc: 0.8882\n",
      "Epoch 2/6\n",
      "20000/20000 [==============================] - 4s 213us/step - loss: 0.2129 - acc: 0.9221 - val_loss: 0.3121 - val_acc: 0.8776\n",
      "Epoch 3/6\n",
      "20000/20000 [==============================] - 4s 223us/step - loss: 0.1482 - acc: 0.9488 - val_loss: 0.3981 - val_acc: 0.8816\n",
      "Epoch 4/6\n",
      "20000/20000 [==============================] - 4s 215us/step - loss: 0.0992 - acc: 0.9655 - val_loss: 0.5022 - val_acc: 0.8726\n",
      "Epoch 5/6\n",
      "20000/20000 [==============================] - 4s 210us/step - loss: 0.0602 - acc: 0.9816 - val_loss: 0.5775 - val_acc: 0.8708\n",
      "Epoch 6/6\n",
      "20000/20000 [==============================] - 4s 217us/step - loss: 0.0402 - acc: 0.9890 - val_loss: 0.7225 - val_acc: 0.8700\n"
     ]
    }
   ],
   "source": [
    "model = models.Sequential()\n",
    "model.add(layers.Dense(32, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dropout(rate=0.5))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='nadam',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "history = model.fit(X_train, y_train,\n",
    "                    epochs=6, batch_size=32,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 在测试集上评估评估模型的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000/25000 [==============================] - 3s 101us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.7677455904477835, 0.85732]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test, y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
